An AI agent might surprisingly find she has reached an unknown state which she has never been aware of -- an unknown unknown. We mathematically ground this scenario in reinforcement learning: an agent, after taking an action calculated from value functions $Q$ and $V$ defined on the {\it {aware domain}}, reaches a state out of the domain. To enable the agent to handle this scenario, we propose an {\it episodic Markov decision {process} with growing awareness} (EMDP-GA) model, taking a new {\it noninformative value expansion} (NIVE) approach to expand value functions to newly aware areas: when an agent arrives at an unknown unknown, value functions $Q$ and $V$ whereon are initialised by noninformative beliefs -- the averaged values on the aware domain. This design is out of respect for the complete absence of knowledge in the newly discovered state. The upper confidence bound momentum Q-learning is then adapted to the growing awareness for training the EMDP-GA model. We prove that (1) the regret of our approach is asymptotically consistent with the state of the art (SOTA) without exposure to unknown unknowns in an extremely uncertain environment, and (2) our computational complexity and space complexity are comparable with the SOTA -- these collectively suggest that though an unknown unknown is surprising, it will be asymptotically properly discovered with decent speed and an affordable cost.

The deployment of autonomous agents in real-world scenarios is challenged by "unknown unknowns", i.e. novel unexpected environments not encountered during training, such as degraded signs. While existing research focuses on anomaly detection and class imbalance, it often fails to address truly novel scenarios. Our approach enhances visual perception by leveraging the Variational Prototyping Encoder (VPE) to adeptly identify and handle novel inputs, then incrementally augmenting data using neural style transfer to enrich underrepresented data. By comparing models trained solely on original datasets with those trained on a combination of original and augmented datasets, we observed a notable improvement in the performance of the latter. This underscores the critical role of data augmentation in enhancing model robustness. Our findings suggest the potential benefits of incorporating generative models for domain-specific augmentation strategies.

As the recent Covid-19 pandemic reminded us, life is filled with unknown unknowns - i.e. contingencies one cannot be aware of ex ante, much less fit into standard risk analysis. In addition to a wealth of examples coming from history and politics, unknown unknowns are now well-documented, and their importance is acknowledged, in many areas of economics and management such as public policy [23], business strategy [5, 11], entrepreneurship [12], contracts and the theory of the firm [40], and security [32]. To be sure, getting hold of such contingencies might allow to achieve significant payoffs or avoid major losses. Substantial research efforts have thus been expended, and notable advances been made, in this direction. To get a rigorous conceptual grasp at the notion of unknown unknowns, one may now draw, notably, from the literatures on Knightian uncertainty (e.g., [4]), undescribable events (e.g., [24]), unforeseen contingencies (e.g., [7, 20]), unawareness (e.g., [35, 34]), and surprises [39, 26]. Yet, for someone who would primarily want to uncover ahead of time the concrete unknown unknowns she might be facing, the task would remain elusive. This paper will now seek to meet this demand.

Student success models might be prone to develop weak spots, i.e., examples hard to accurately classify due to insufficient representation during model creation. This weakness is one of the main factors undermining users' trust, since model predictions could for instance lead an instructor to not intervene on a student in need. In this paper, we unveil the need of detecting and characterizing unknown unknowns in student success prediction in order to better understand when models may fail. Unknown unknowns include the students for which the model is highly confident in its predictions, but is actually wrong. Therefore, we cannot solely rely on the model's confidence when evaluating the predictions quality. We first introduce a framework for the identification and characterization of unknown unknowns. We then assess its informativeness on log data collected from flipped courses and online courses using quantitative analyses and interviews with instructors. Our results show that unknown unknowns are a critical issue in this domain and that our framework can be applied to support their detection. The source code is available at https://github.com/epfl-ml4ed/unknown-unknowns.

Artificial Intelligence (AI) is opening a wonderful world of immense possibilities. Such prospects include, for example, helping save the Amazon by forecasting deforestation; automation and job creation through reskilling; mitigating and managing climate change by measuring emissions; boosting the discovery of new drugs; fighting terrorism and transforming national security; and improving criminal justice system and cutting crime rates. AI-based autonomous vehicles -- cars, trucks, buses and drone delivery systems -- are already impacting our lives. By using AI, metropolitan areas could be transformed into smart cities for service delivery, environment planning, power utilization, handling emergencies and much more. These are some of the known and knowable problems that the applications of AI algorithms can solve with greater efficiency.

The cybersecurity industry is rapidly embracing the notion of "zero trust", where architectures, policies, and processes are guided by the principle that no one and nothing should be trusted. However, in the same breath, the cybersecurity industry is incorporating a growing number of AI-driven security solutions that rely on some type of trusted "ground truth" as reference point. This is not a hypothetical discussion. Organizations are introducing AI models into their security practices that impact almost every aspect of their business, and one of the most urgent questions remains whether regulators, compliance officers, security professionals, and employees will be able to trust these security models at all. Because AI models are sophisticated, obscure, automated, and oftentimes evolving, it is difficult to establish trust in an AI-dominant environment.

Imagine visiting the control room of a metals company. You're there to discuss asset performance and process optimization. During the visit, you see on a desk a computer mouse wrapped in a rubber band. On the nearby computer screen, the cursor hovers over an icon that a person would click to acknowledge an alarm triggered by the automated system tracking the thousands of sensors placed throughout the company's facilities. It seems it would never be clear to the person sitting in that chair if there was a serious problem or not.

Recent work has demonstrated the promise of combining local explanations with active learning for understanding and supervising black-box models. Here we show that, under specific conditions, these algorithms may misrepresent the quality of the model being learned. The reason is that the machine illustrates its beliefs by predicting and explaining the labels of the query instances: if the machine is unaware of its own mistakes, it may end up choosing queries on which it performs artificially well. This biases the "narrative" presented by the machine to the user. We address this narrative bias by introducing explanatory guided learning, a novel interactive learning strategy in which: i) the supervisor is in charge of choosing the query instances, while ii) the machine uses global explanations to illustrate its overall behavior and to guide the supervisor toward choosing challenging, informative instances. This strategy retains the key advantages of explanatory interaction while avoiding narrative bias and compares favorably to active learning in terms of sample complexity. An initial empirical evaluation with a clustering-based prototype highlights the promise of our approach.

In conventional supervised learning, a training dataset is given with ground-truth labels from a known label set, and the learned model will classify unseen instances to the known labels. In this paper, we study a new problem setting in which there are unknown classes in the training dataset misperceived as other labels, and thus their existence appears unknown from the given supervision. We attribute the unknown unknowns to the fact that the training dataset is badly advised by the incompletely perceived label space due to the insufficient feature information. To this end, we propose the exploratory machine learning, which examines and investigates the training dataset by actively augmenting the feature space to discover potentially unknown labels. Our approach consists of three ingredients including rejection model, feature acquisition, and model cascade. The effectiveness is validated on both synthetic and real datasets.

Machine learning as a way of managing a portfolio is a "wave of the future" with a lot of sea to cover before it breaks on the shore. Meson Capital Partners has sent its investors an update and summary for 2019 Q3 regarding its managed market neutral fund, and this letter illustrates the forward motion of machine learning. Following a recent restructure, all Meson investments are now in liquid public securities. Management vows that any future illiquid investments will be confined to investment vehicles set up for such assets. Meson's liquid portfolio is about 70% managed by its machine-learning platform, Gravity Technologies.